Scalable service level agreement (sla) verification and action using a data plane

ABSTRACT

An ingress node inserts into a header of a packet service level agreement information and forwards the packet. At an egress node of the network, the packet is received and the service level agreement information is obtained from the header of the packet. The egress node verifies whether there is conformance to a service level agreement based on at least one parameter associated with reception of one or more packets at the egress node and the service level agreement information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/219,678, filed Jul. 26, 2016, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to inband service level agreement (SLA)verification.

BACKGROUND

Network service providers are expected to provide differentiated levelsof service according to service level agreements (SLAs). Currently, SLAverification measures latency, including jitter and delay, by using anout-of-band mechanism and sending synthetic probe packets that areindependent of production data flows. While an effort is made toresemble production traffic, such as voice or video, an actual path thatthe production traffic takes, based on hashing, could differ from a pathof out-of-band traffic. Latency could be related to a characteristic ofa specific flow, a specific path, a specific size, etc., which may notbe detected by an out-of-band probe packet. Consequently, during SLAverification, an exact service level achieved by a specific user may notbe determined.

Network analytics is a key area for monetization of a network. Forexample, the NetFlow™ technology is used for collecting network and flowrelated information, which is periodically uploaded to a networkmanagement server for analytics. The analytics may be used for real-timeservice assurance. Federating such distributed data in order to performreal-time and predictive service assurance is difficult to achieve on aper-flow basis. For ease and scalability, real-time and predictiveservice assurance checks for any resource constraints, and if apredefined threshold is breached, a link/path will be excluded. However,per-flow visibility is not possible.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a network environment in which theinband SLA verification techniques presented herein may be employed,according to an example embodiment.

FIG. 2 illustrates an example ingress node configured to participate inthe inband SLA verification techniques, according to an exampleembodiment.

FIG. 3 shows an example egress node configured to participate in theinband SLA verification techniques, according to an example embodiment.

FIG. 4 illustrates SLA information carried inband in an InternetProtocol, version 6, header, according to an example embodiment.

FIG. 5 illustrates SLA information carried inband in a network serviceheader, according to an example embodiment.

FIG. 6 is a flowchart depicting operations performed in an ingress node,according to an example embodiment.

FIG. 7 is a flowchart depicting operations performed in an egress node,according to an example embodiment.

FIGS. 8 and 9 illustrate a Multiprotocol Label Switched (MPLS) labelwith a particular value used to trigger generation and sending of anetwork resource report to a server, according to an example embodiment.

FIG. 10 is similar to FIG. 9, but illustrates use of an IP version 6extension header with a NetFlow label to trigger a network resourcereport, according to an example embodiment.

FIG. 11 is a flowchart of a process in which a packet tagged with aNetFlow label triggers generation and transmission of a network resourcereport, according to an example embodiment.

DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

An ingress node inserts into a header of a packet service levelagreement information and forwards the packet. At an egress node of thenetwork, the packet is received and the service level agreementinformation is obtained from the header of the packet. The egress nodeverifies whether there is conformance to a service level agreement basedon at least one parameter associated with reception of one or morepackets at the egress node and the service level agreement information.

Example Embodiments

FIG. 1 illustrates an example of an environment 100 in which the variousembodiments presented herein may be deployed. The environment includes anetwork 102 of which there is an ingress node 104 and egress node 106.There are numerous other nodes in the network 102 that are not shown forsimplicity. According to embodiments presented herein, the ingress node104 generates packets according to flow information and inserts expectedor agreed-upon SLA information into headers of one or more packets of apacket flow. The expected SLA information may include agreed-upon delay(latency), jitter, packet loss, etc. Delay (latency) is measured interms of end-to-end (ingress-to-egress) time taken for a packet totravel. Jitter is a variation on delay and measures the delay overvarious packets, and compares the delay differences. Packet loss is ameasure of drops of packets, and is commonly reflected as a percentage(packet loss percentage). In accordance with the techniques presentedherein, the egress node will measure delay/latency for packets. Theegress node can additionally compute jitter (over a plurality ofpackets) as well as packet loss by comparing an in-packet carriedsequence number value to an expected sequence number of a receivedpacket. In any case, the expected SLA information (whether it is delay,jitter, packet loss, etc.) is inserted into at least one packet that issent from the ingress node into the network and for reception by theegress node.

The flow information may be 5-tuple-based (e.g., source IP address/portnumber, destination IP address/port number and the protocol of aTransmission Control Protocol/Internet Protocol (TCP/IP) connection),Quality of Service (QoS)-based (precedence/Differentiated Services CodePoint (DSCP)), ingress interface-based, or Virtual Routing andForwarding (VRF)-based, etc.

The ingress node 104 may receive a packet that is part of a flow thatmatches a SLA requirement policy (i.e., a packet that is part of a flowto be monitored for SLA compliance) and may append to the packet, aspart of a packet header, the expected SLA information. In embodimentsthat operate in an IP version 6 environment, the packet header may be anextension header. In other embodiments, the packet header may be anetwork service header. The ingress node 104 may then forward the packetinto network 102 where it eventually reaches the egress node 106.

The above-described actions of the ingress node 104 could be consideredas a Service Function Chaining paradigm where the egress node 106 servesas a Service Function Forwarder hosting a Service Function that isresponsible for performing network analytics such as, for example, SLAverification.

When the egress node 106 receives the packet and determines, based onthe expected SLA information in the header, that the packet failed SLAverification, the egress node 106 may send a notification to the ingressnode 104 and/or to a network controller 108 indicating that the packetfailing the SLA verification. The notification signaling may beaccomplished by using either an out-of-band mechanism or an inbandOperation, Administration and Maintenance (OAM) mechanism.

Upon receiving the notification, the ingress node 104 or the networkcontroller 108 could take a number of different actions, some of whichmay include, but not be limited to, triggering a Fast Retransmit andRecovery (FRR) or increasing a metric on a Provider Edge (PE) toCustomer Edge (CE) link to redirect traffic.

In some embodiments, the notification may be sent to network controller108, or to a centralized management server (not shown) for accountingpurposes.

Integration with application endpoints is also possible. For example,the egress node 106 or the network controller 108, upon knowing anapplication within a flow, may signal an originating endpoint or acontroller of the originating endpoint (for a voice or video call) tothrottle transmission, change codecs, or perform another action. Furtherdetails and use cases will become more apparent in connection with thefollowing description.

FIG. 2 illustrates a block diagram of an ingress node, shown atreference numeral 200. The ingress node 200 includes input ports 210 andoutput ports 220. In one embodiment, the ingress node 200 includes anetwork processor Application Specific Integrated Circuit (ASIC) 230, ormultiple such ASICs. Network processor ASIC 230 may include fixeddigital logic, programmable logic, or a combination thereof. Forexample, network processor ASIC 230 may include fixed or programmabledigital logic integrated circuits, in which digital logic gates areconfigured to perform instructions of SLA information insertion logic240. Network processor ASIC 230 may further include memory (not shown)and fixed or programmable digital logic for networking functions, suchas switching, routing, etc.

The ingress node 200 may include network processor ASIC 230 or memory250 or both network processor ASIC 230 and memory 250. The ingress node200 may further include one or more processors 270. Memory 250 mayinclude SLA information insertion logic 260.

The one or more processors 270 may be one or more microprocessors ormicrocontrollers configured to execute program logic instructions, suchas SLA information insertion logic 260, for carrying out variousoperations and tasks described herein. For example, the one or moreprocessors 270 may execute SLA information insertion logic 260 stored inmemory 250 (as e.g., software) in order to perform SLA informationinsertion techniques described herein. Memory 250 may include read onlymemory (ROM), random access memory (RAM), magnetic storage media,optical storage media, flash memory, electrical, or otherphysical/tangible (non-transitory) memory.

The functions of one or more processors 270 may be implemented by logicencoded in one or more non-transitory tangible computer readable media,wherein memory 250 may store data used for operations described hereinand may store software or processor executable instructions that areexecuted to carry out the operations described herein.

SLA information insertion logic 260 may take any of a variety of forms,so as to be encoded in one or more non-transitory tangible computerreadable memory media or storage device (e.g., memory 250) forexecution, such as fixed logic or programmable logic (e.g.,software/computer instructions executed by a processor).

FIG. 3 illustrates a block diagram of an egress node shown at referencenumeral 300. The egress node 300 may include input ports 310 and outputports 320. In one embodiment, the egress node 300 includes a networkprocessor ASIC 330, or multiple such ASICs. Network processor ASIC 330may include fixed digital logic, programmable logic, or a combinationthereof. For example, network processor ASIC 330 may include fixed orprogrammable digital logic integrated circuits, in which digital logicgates are configured to perform instructions of SLA verification logic340. Network processor ASIC 330 may further include memory (not shown)and fixed or programmable digital logic for networking functions, suchas switching, routing, etc.

The egress node 300 may include network processor ASIC 330 or memory 350or both network processor ASIC 330 and memory 350. The egress node 300may further include one or more processors 370. Memory 350 may includeSLA verification logic 360.

The one or more processors 370 may be one or more microprocessors ormicrocontrollers configured to execute program logic instructions suchas SLA verification logic 360 for carrying out various operations andtasks described herein. For example, one or more processors 370 canexecute SLA verification logic 360 stored in memory 350 (as e.g.,software) in order to perform SLA verification techniques describedherein. Memory 350 may include ROM, RAM, magnetic storage media, opticalstorage media, flash memory, electrical, or other physical/tangible(non-transitory) memory.

The functions of one or more processors 370 may be implemented by logicencoded in one or more non-transitory tangible computer readable media,wherein memory 350 may store data used for operations described hereinand may store software or processor executable instructions that areexecuted to carry out the operations described herein.

SLA verification logic 360 may take any of a variety of forms, so as tobe encoded in one or more non-transitory tangible computer readablememory media or storage device (e.g., memory 350) for execution, such asfixed logic or programmable logic (e.g., software/computer instructionsexecuted by a processor).

FIGS. 4 and 5 illustrate example topologies for various embodiments ofthe inband SLA verification techniques. FIG. 4 illustrates a topology inan IP version 6 environment. FIG. 5 illustrates a topology in anenvironment that uses a network service header. FIGS. 4 and 5 depict anexample in which the SLA expectation information is related tolatency/delay. This is only by way of example and is not meant to belimiting.

In FIG. 4, a network 400 is shown that includes an ingress node 402. Theingress node 402 receives a packet 404 that is part of a flow for whichSLA verification is to be made. The packet 404 includes source addressinformation, destination address information and a payload. Upondetermining that the received packet 404 matches the SLA requirementpolicy, the ingress node 402 may append an IP version 6 extension header406 to the packet 404. The extension header may include SLA expectationinformation and a first local timestamp indicating a time of departureof the packet from the ingress node. For example, the SLA expectationinformation is a latency of 80 msec and the first local timestamp is T1.After appending the IP version 6 extension header 406 to the packet 404,the ingress node 402 may forward the packet 404 into network 400 whereit ultimately will reach an egress node. As explained above, appendingthe IP version 6 extension header and forwarding the packet 404 to anegress node 408 may be considered to be Service Function Chaining (SFC)with the egress node 408 as a Service Function Forwarder (SFF) hosting aService Function (SF) that is responsible for performing networkanalytics such as, for example, SLA verification.

Upon receiving the packet 404 with the appended extension header 406,the egress node 408 obtains a second local timestamp, T2, the time atwhich the packet 404 arrives at the egress node, and verifies whetherthe packet conforms to SLA expectation information based on a differencebetween the second local timestamp and the first local timestamp and theSLA expectation. In other words, the egress node uses T2 as a localprocessing timestamp and determines the difference between T1 and T2 forthe packet to determine if the end-to-end (ingress-to-egress)delay/latency is within the SLA expectation, e.g., 80 msec, that wasinserted into the packet.

When the egress node 408 determines that the packet having the appendedextension header is not verified as conforming to the SLA, the egressnode 408 may send a notification indicating a failed SLA verification tothe ingress node 402 or to a network controller. The notification may besent via an out-of-band mechanism or an inband OAM mechanism.

Upon receiving the notification, the ingress node 402 or the networkcontroller could take a number of different actions including, but notlimited to, triggering a Fast Retransmit and Recovery or increasing ametric on a PE to CE link to redirect traffic.

Reference is now made to FIG. 5. FIG. 5 shows a network 500 having aningress node 502. The ingress node 502 receives a packet 504 that ispart of a flow for which SLA verification is to be made. The packet 504includes source address information, destination address information anda payload. Upon determining that the received packet 504 matches an SLArequirement policy, the ingress node 502 may append a network serviceheader (NSH) 506 to the packet 504. The network service header mayinclude SLA expectation information and a first local timestamp. Again,in this example, the SLA expectation information is a latency of 80 msecand the first local timestamp is T1. After appending the network serviceheader 506 to the packet 504, the ingress node 502 may forward thepacket 504 into the network 50 where it ultimately reaches egress node508.

Upon receiving the packet 504 with the appended network service header506, the egress node 508 obtains a second local timestamp (T2) andverifies whether the packet conforms to the SLA based on a differencebetween the second local timestamp and the first local timestamp and theSLA expectation information, as described above in connection with FIG.4.

As described above in connection with FIG. 4, when the egress node 508determines that the packet having the appended network service header isnot verified as conforming to the SLA, the egress node 508 may send anotification indicating a failed SLA verification to the ingress node502 or to a network controller. The notification may be sent via anout-of-band mechanism or an inband OAM mechanism.

Upon receiving the notification, the ingress node 502 or the networkcontroller could take a number of different actions including, but notlimited to, triggering a Fast Retransmit and Recovery or increasing ametric on a PE to CE link to redirect traffic.

SLA verification, using either IP version 6 extension headers or networkservice headers, may be integrated with application endpoints. In suchembodiments, upon receiving the signal, the ingress node 402, 502 or thenetwork controller may throttle transmission, change codecs, or mayperform another action.

FIG. 6 is a flowchart that illustrates example processing in variousembodiments of an ingress node. At 602, the process may begin with theingress node receiving or obtaining a packet via one of its input ports.The ingress node may then determine, at 604, whether the packet matchesan SLA requirement policy. If the ingress node determines that thepacket does not match the SLA requirement policy, then the packet may beforwarded into the network at 614.

At 604, if the ingress node determines that the packet matches the SLArequirement policy, then a header may be appended to the packet. Theheader may be an IP version 6 extension header in some embodiments andin other embodiments, the header may be a network service header.

At 608, the ingress node may copy/insert SLA information into the headerappended to the packet. The SLA information may include an SLAexpectation, such as latency, jitter or packet loss, for example.

When the SLA information pertains to delay or jitter, operations 610 and612 are performed. At 610, the ingress node obtains a first localtimestamp and, at 612, the ingress node may include the first localtimestamp in the header. The first local timestamp represents the timethat the packet is sent into the network by the ingress node.

The ingress node may then forward the packet with the appended headerinto the network, at 614. If the SLA information pertains to a parameterother than delay or jitter, such as packet loss, then operations 610 and612 are not necessary, but sequence numbers are included in packets sothat the egress node can determine when packets that are expected to bereceived (based on sequence number), are not received. This is usefulfor determine conformance with a SLA-based packet loss.

FIG. 7 is a flowchart illustrating example processing in the egressnode. At 702, the egress node receives the packet (which originated atthe ingress node) in the network. At 704, the egress node determineswhether the packet includes a header (an IP version 6 extension headeror a network service header) having stored therein SLA expectationinformation (and perhaps a local timestamp). If the egress nodedetermines that the packet does not include the header having storedtherein the SLA expectation information then, at 706, the packet isforwarded to a destination and the process is completed.

If the egress node determines that the packet includes the header havingstored therein SLA expectation information and a local timestamp, thenthe following operations are performed. When the header further a localtimestamp inserted by the ingress node, then at 708, the egress node mayobtain a second local timestamp associated with receipt of the packet atthe egress node. At 710, the egress node obtains the SLA informationcontained in the packet and the first local timestamp, if one isincluded. At 712, the packet may be forwarded to the destination.

At 714, the egress node may submit the SLA information to a localprocess (running at the egress node) to determine whether there isconformance to the SLA based on the SLA information obtained from theheader of the packet.

The determination made at 714 depends on the type of SLA informationcontained in the header of the received packet. When the header of thereceived packet includes a first local timestamp inserted by the ingressnode and the SLA information contained in the header is expectedlatency/delay, then the egress node uses the second timestamp (obtainedat 708) associated with reception of the packet at the egress node, anddetermines a difference between the first local timestamp and the secondlocal timestamp. Thus, in this example, verification of conformance tothe SLA involves comparing the difference with the expected latencyvalue contained in the header of the packet. If the difference isdetermined to be outside the expected latency, then the conformance ofthe packet to the SLA fails.

When the SLA information in the header of the packet is expected jitter,then the egress node determines differences, for a plurality of packetsreceived at the egress node, between a first local timestamp associatedwith a packet sent by the ingress node and a second local timestampassociated with a packet received at the egress node. The egresscomputes a jitter value based on the differences, and compares thejitter value with the expected jitter. If the jitter value is outsidethe expected jitter, then conformance to the SLA fails.

Further still, when the SLA Information in the header of the packet isexpected packet loss, the egress node determines whether and how manypackets sent by the ingress node do not reach the egress node based onsequence number information contained in the packets, and the egressnode computes a packet loss value, accordingly. Then egress node thencompares the packet loss value with the expected packet loss, and whenthe packet loss value is outside the expected packet loss, conformanceto the SLA fails.

When conformance to the SLA fails, a packet counter may be incremented,at 718. If, at 720, the packet counter is determined not to be greaterthan or equal to a predefined maximum count, then the process iscompleted. Otherwise, the packet counter may be reset, at 722, and asignal may be sent, at 724.]

As mentioned previously, the signal may be sent to the ingress node or anetwork controller. In some embodiments, in which the egress node or thenetwork controller has knowledge of an application within the flow, theegress node or the network controller may signal an originating endpointof the application, or a network controller of the originating endpoint,to perform an action including, but not limited to, throttlingtransmission or changing a codec.

In some embodiments, in addition to the packet counter being reset whenthe packet counter becomes greater than or equal to the predefinedmaximum count, the packet counter may also be reset after eachpredefined time interval. The predefined time interval may be 10seconds, 60 seconds, or some other time interval. In such embodiments, asignal indicating failed SLA verification will be sent every time thepacket counter reaches the predefined maximum count during a predefinedtime interval.

Embodiments which use inband SLA verification provide a clear pictureregarding whether actual data traffic is flowing according to theexpected SLA. Further, SLA verification is very granular such that itcould be used as: always on; selective packet based verification; oron-demand verification. As explained above, SLA verification may bebased on delay (latency) measured in terms of end-to-end(ingress-to-egress) time difference of a packet, as well as jitter andpacket loss. The egress node will measure delay/latency for packets. Theegress node can additionally compute jitter (over a plurality ofpackets) as well as packet loss by comparing an in-packet carriedsequence number value to an expected sequence number of a receivedpacket. Thus, the SLA information inserted into a header of a packet atan ingress node may include expected/maximum delay/latency,expected/maximum jitter and expected/maximum packet loss.

Thus, in summary, in one form, a method is provided comprising:receiving, at an egress node of a network, a packet that includes in aheader thereof service level agreement information inserted by aningress node of the network; obtaining, by the egress node, the servicelevel agreement information; and verifying, by the egress node, whetherthere is conformance to a service level agreement based on at least oneparameter associated with reception of one or more packets and theservice level agreement information.

In another form, an apparatus is provided comprising: a plurality ofnetwork ports of an egress node configured to receive and send packetsover a network; a network processor unit coupled to the plurality ofnetwork ports and configured to determine how to direct packets withrespect to the plurality of network ports; and a processor configuredto: obtain, at an egress node of a network, from a header of a packetreceived at one of the plurality of network ports, service levelagreement information inserted by an ingress node of the network, andverify whether the packet conforms to a service level agreement based onat least one parameter associated with reception of one or more packetsand the service level agreement information.

In still another form, one or more non-transitory computer readablestorage media are provided that are encoded with instructions, whichwhen executed by a processor, cause the processor to perform operationscomprising: receiving, at an egress node of a network, a packet thatincludes in a header thereof service level agreement informationinserted by an ingress node of the network; obtaining, by the egressnode, the service level agreement information; verifying, by the egressnode, whether there is conformance to a service level agreement based onat least one parameter associated with reception of one or more packetsand the service level agreement information.

In yet another form, a method is provided comprising: obtaining, at aningress node of a network, a packet to be sent into the network;inserting into a header of the packet service level agreementinformation for use by an egress node of the network in verifyingconformance to a service level agreement; and forwarding the packet intothe network.

In still another form, an apparatus is provided comprising: a pluralityof network ports of an ingress node configured to receive and sendpackets over a network; a network processor unit coupled to theplurality of network ports and configured to determine how to directpackets with respect to the plurality of network ports; and a processorconfigured to: obtain, at an ingress node of a network, a packet to besent into the network; insert into a header of the packet service levelagreement information for use by an egress node of the network inverifying conformance to a service level agreement; and forward thepacket into the network.

In still another form, one or more non-transitory computer readablestorage media are provided that are encoded with instructions, whichwhen executed by a processor, cause the processor to perform operationscomprising: obtaining, at an ingress node of a network, a packet to besent into the network; inserting into a header of the packet servicelevel agreement information for use by an egress node of the network inverifying conformance to a service level agreement; and forwarding thepacket into the network.

In yet another form, a method is provided comprising: obtaining, at aningress node of a network, a packet to be sent into the network;inserting into a header of the packet service level agreementinformation for use by an egress node of the network in verifyingconformance to a service level agreement; forwarding the packet into thenetwork; receiving, at the egress node the packet that includes in theheader thereof service level agreement information inserted by theingress node; obtaining, by the egress node, the service level agreementinformation; and verifying, by the egress node, whether there isconformance to a service level agreement based on at least one parameterassociated with reception of one or more packets and the service levelagreement information.

In still another form, a system is provided comprising an ingress node,an egress node, both of which are part of a network, wherein the ingressnode is configured to: obtain a packet to be sent into the network;insert into a header of the packet service level agreement informationfor use by the egress node of the network in verifying conformance to aservice level agreement; and forward the packet into the network; andwherein the egress node is configured to: receive the packet thatincludes in the header thereof service level agreement informationinserted by the ingress node; obtain the service level agreementinformation; and verify whether there is conformance to a service levelagreement based on at least one parameter associated with reception ofone or more packets and the service level agreement information.

Label-Based Source-Triggered NetFlow and Resource Data Collection

Typically, network devices such as, for example, routers, are configuredto collect and export network information. However, per-flow visibilityis not generally provided. To solve this problem, control is placed on asource instead of an intermediate router. For example, an instructionmay be included in a packet instead of in a router command lineinterface (CLI). This can be achieved by tagging a packet with aspecific marking. In one embodiment, this can be accomplished by using aMultiprotocol Label Switching (MPLS) label value (“NetFlow Label” orNFL). The NFL is applicable to traditional MPLS networks and segmentrouting. In another embodiment, this can be achieved by using an IPversion 6 Flow Label value or an IP version 6 extension header signal.

Reference is now made to FIGS. 8 and 9. FIG. 8 shows an embodiment usingMPLS labels in this manner. A label table 800 of a label switch router(LSR) or transit router is shown. A value, 1010 in this example, isassociated with a “NetFlow Query” behavior. Therefore, when a labelhaving the value 1010 arrives at a transit router, in addition toforwarding a packet as usual, a network resource report is triggered andsent to a server 902, as shown in FIG. 9. The network resource reportmay include information such as local resource utilization of an ingressbuffer queue, ingress card CPU utilization, ingress memory utilization,etc.

Although FIG. 8 depicts an embodiment using MPLS labels, the sameconcept may be used with respect to another embodiment that uses, forexample, IP version 6 extension headers. In other words, the concept maybe used with any way of tagging a packet.

FIG. 10 shows an example similar to that of FIG. 9, but using an IPversion 6 extension header with NFL 1014 instead of an MPLS label.

FIG. 11 illustrates an example process in which packets in a flow may betagged with a NFL to trigger a network resource report. The process maybegin, at 1102, with an ingress node tagging a packet with a NFL andforwarding the packet. At 1104, a node receives the packet. The node maydetermine whether the packet is tagged with the NFL, at 1106. If thenode is not tagged with the NFL, then the packet is forwarded, at 1112,and the process may be completed. Otherwise, the node forwards thepacket, at 1108, and at 1110, triggers creation of a network resourcereport, which then may be sent to a central server. The process may thenbe completed.

The sending of network reports is scalable because every packet may notinclude a NFL. For example, the NFL may be included in every N^(th)packet, where N may be 100, 50, 200, or another number.

The central server may use network resource information to performanalytics and may perform an action if the SLA is breached or predictedto be breached based on the collected network resource information.

In some embodiments, by default, information captured by NetFlow anduploaded for real-time analytics may include: flow information such as,for example, a header tuple, protocol, etc.; packet context, which mayinclude path information such as incoming interface, outgoing interface,stack details (e.g., MPLS or segment routing (SR) stack); and networkcontext, which may include resource information such as CPU utilization,backplane utilization, etc.

In its most fundamental form, a static generic tag may be usedincluding, but not limited to an MPLS label, an IP version 6 flow label,an extension header, and a Type-Length Value (TLV) within inband OAMwithin IP version 6 (iOAM6).

Positioning a well-defined and unique tag anywhere in a stack to triggercollection of NetFlow information introduces a new paradigm indiagnostics collection. The paradigm shifts from adevice/configuration-based paradigm to a more scalable packet-basedparadigm and moves control from transit devices to a source, where thereis flexibility to perform a full rate/bulk collection to selectivepacket marking for custom or in-depth flow analysis.

The above description is intended by way of example only. Variousmodifications and structural changes may be made therein withoutdeparting from the scope of the concepts described herein and within thescope and range of equivalents of the claims.

What is claimed is:
 1. A method comprising: obtaining, at an ingressnode of a network, a packet to be sent into the network; inserting intoa header of the packet, service level agreement information for use byan egress node of the network in verifying conformance to a servicelevel agreement; forwarding the packet into the network; obtaining, fromthe egress node, a notification indicating that verification of theservice level agreement has failed; and in response to obtaining thenotification, taking an action for subsequent packets obtained by theingress node.
 2. The method of claim 1, further comprising: determiningwhether the packet is part of a flow to be monitored for service levelagreement conformance; wherein the inserting is performed when thepacket is determined to be part of the flow to be monitored for theservice level agreement conformance.
 3. The method of claim 1, whereinthe header is an Internet Protocol extension header or a network serviceheader.
 4. The method of claim 1, wherein the service level agreementinformation includes one or more of: expected latency, expected jitter,and expected packet loss.
 5. The method of claim 4, further comprisingobtaining, by the ingress node, a first local timestamp associated witha time that the packet is sent into the network by the ingress node foruse in enabling the egress node to determine latency or jitter.
 6. Themethod of claim 1, wherein taking an action for subsequent packetsobtained by the ingress node includes triggering a Fast Retransmit andRecovery.
 7. The method of claim 1, wherein taking an action forsubsequent packets obtained by the ingress node includes increasing ametric on a link between a provider edge device and a customer edgedevice to redirect traffic.
 8. An apparatus comprising: a processor; anda memory to store data and instructions executable by the processor,wherein the processor is configured to execute the instructions to:obtain, at an ingress node of a network, a packet to be sent into thenetwork; insert into a header of the packet service level agreementinformation for use by an egress node of the network in verifyingconformance to a service level agreement; forward the packet into thenetwork; obtain, from the egress node, a notification indicating thatverification of the service level agreement has failed; and in responseto obtaining the notification, take an action for subsequent packetsobtained by the ingress node.
 9. The apparatus of claim 8, wherein theprocessor is configured to execute the instructions to: determinewhether the packet is part of a flow to be monitored for service levelagreement conformance; and insert the service level agreementinformation into the header of the packet when the processor determinesthat the packet is to be part of the flow to be monitored for theservice level agreement conformance.
 10. The apparatus of claim 8,wherein the header is an Internet Protocol extension header or a networkservice header.
 11. The apparatus of claim 8, wherein the service levelagreement information includes one or more of: expected latency,expected jitter, and expected packet loss.
 12. The apparatus of claim11, wherein the processor is configured to execute the instructions to:obtain, by the ingress node, a first local timestamp associated with atime that the packet is sent into the network by the ingress node foruse in enabling the egress node to determine latency or jitter.
 13. Theapparatus of claim 8, wherein the processor is configured to take anaction for subsequent packets obtained by the ingress node by triggeringa Fast Retransmit and Recovery.
 14. The apparatus of claim 8, whereinthe processor is configured to take an action for subsequent packetsobtained by the ingress node by increasing a metric on a link between aprovider edge device and a customer edge device to redirect traffic. 15.One or more non-transitory computer-readable storage media encoded withsoftware comprising computer executable instructions which, whenexecuted by a processor, cause the processor to: obtain, at an ingressnode of a network, a packet to be sent into the network; insert into aheader of the packet service level agreement information for use by anegress node of the network in verifying conformance to a service levelagreement; forward the packet into the network; obtain, from the egressnode, a notification indicating that verification of the service levelagreement has failed; and in response to obtaining the notification,take an action for subsequent packets obtained by the ingress node. 16.The one or more non-transitory computer-readable storage media of claim15, wherein the instructions cause the processor to: determine whetherthe packet is part of a flow to be monitored for service level agreementconformance; and insert the service level agreement information into theheader of the packet when the processor determines that the packet is tobe part of the flow to be monitored for the service level agreementconformance.
 17. The one or more non-transitory computer-readablestorage media of claim 15, wherein the header is an Internet Protocolextension header or a network service header.
 18. The one or morenon-transitory computer-readable storage media of claim 15, wherein theservice level agreement information includes one or more of: expectedlatency, expected jitter, and expected packet loss.
 19. The one or morenon-transitory computer-readable storage media of claim 18, wherein theinstructions cause the processor to: obtain, by the ingress node, afirst local timestamp associated with a time that the packet is sentinto the network by the ingress node for use in enabling the egress nodeto determine latency or jitter.
 20. The one or more non-transitorycomputer-readable storage media of claim 15, wherein the instructionscause the processor to take an action for subsequent packets obtained bythe ingress node by triggering a Fast Retransmit and Recovery orincreasing a metric on a link between a provider edge device and acustomer edge device to redirect traffic.